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Abstract 

This paper presents a monoidal category whose morphisms are games 
(in the sense of game theory, not game semantics) and an associated 
diagrammatic language. The two basic operations of a monoidal category, 
namely categorical composition and tensor product, correspond roughly 
to sequential and simultaneous composition of games. This leads to a 
compositional theory in which we can reason about properties of games in 
terms of corresponding properties of the component parts. In particular, 
we give a definition of Nash equilibrium which is recursive on the causal 
structure of the game. 

The key technical idea in this paper is the use of continuation pass¬ 
ing style for reasoning about the future consequences of players’ choices, 
closely based on applications of selection functions in game theory. Ad¬ 
ditionally, the clean categorical foundation gives many opportunities for 
generalisation, for example to learning agents. 


1 Introduction 

This paper presents a monoidal category whose morphisms are games (in the 
sense of game theory, not game semantics) and an associated diagrammatic lan¬ 
guage. The two basic operations of a monoidal category, namely categorical 
composition and tensor product, correspond roughly to sequential and simul¬ 
taneous composition of games. This leads to a compositional theory in which 
we can reason about properties of games in terms of corresponding properties 
of the component parts. In particular, we give a definition of Nash equilibrium 
which is recursive on the causal structure of the game. 

The context of this paper begins with |19] . which for the first time ap¬ 
proached game theory using ideas from program semantics. Many ideas used 
in this paper first appear there, such as the idea that a game should be a ‘pro¬ 
cess’ mapping observations to choices, and the use of monads (or equivalently 
premonoidal categories) to model game-theoretic side effects such as probabilis¬ 
tic choice. Nevertheless, the game theory developed in that paper is no more 
compositional than ordinary game theory. 

Giving a compositional theory of games is far from straightforward. The 
main objects of study in this paper are so-called pregames, which can be thought 
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of as games relative to a continuation. The use of continuation passing style in 
game theory is implicit in the literature on selection functions (see [8] for an 
introduction), and was recently made explicit by the author in an (unpublished) 
extended abstract m- Pregames are also closely related to the ‘partially de¬ 
fined games’ of m Broadly speaking, when a player computes which move to 
play, she does so with knowledge about how that move is going to be used by 
the environment (consisting of the rules of the game and the other players) to 
compute an outcome. If we take this intuition seriously and relativise an entire 
game to a continuation, including the equilibrium condition, it becomes possible 
to develop a compositional theory of games. 

The use of string diagrams as an internal language for monoidal categories 
has been developed in quantum information theory [4] and bialgebra [lOj . and 
also applied in linguistics [6]. A string diagram denotes a morphism of a 
monoidal category, and the connection is made formal by a coherence theo¬ 
rem saying that the morphism denoted by a string diagram is invariant under 
certain topological moves of the diagram. There are many variants of monoidal 
categories, each with its own associated string language, surveyed in [^. The 
language used in this paper is the one for symmetric monoidal categories (sec¬ 
tion 3.5 of loc. cit.), extended with an operation called teleological unit, which 
is unique to game theory and developed in this paper. 

Our string diagrams appear to be related to influence diagrams, an existing 
graphical language applied mostly in decision theory but also in game the¬ 
ory m- Although the precise relationship is still to be worked out, string 
diagrams appear to be preferable because their well-understood categorical se¬ 
mantics (symmetric monoidal categories) allows them to be easily generalised 
beyond ordinary probabilistic choices. 

The notation used in this paper is intentionally reminiscent of the notation 
used in linear algebra, quantum theory and linguistics. The categories appear¬ 
ing in those areas have additional structure (namely, a compact closed or dagger 
structure) which gives a ‘quantum causality’ in which information can appear 
to flow both forwards and backwards in time. Although we do not have this 
structure, the teleological unit appears very similar to the unit of a compact 
structure, and in particular satisfies a coherence theorem very similar to nat- 
urality of the unit. The game-theoretic interpretation of this is that we have 
a limited form of backward-causality due to rational agents reasoning about 
future values. 

Outline Section [2] defines pregames, the objects of study of this paper, defines 
the various categorical operations on them, and introduces the string diagram 
language. Section |3] proves the coherence theorems necessary for the string 
language to be well-defined. Section |4] details the relationship between pregames 
and selection functions, on which they are based. Finally section [5] gives several 
directions for applications and theoretical research. 
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Conventions In this paper we work only with pure strategies, to simplify 
the proofs in section [S] This amounts to defining our constructions starting 
from the category of sets, but all of the theory in this paper works for an 
arbitrary symmetric monoidal category, of which the most obvious to use is the 
category of stochastic relations m- Other possibilities, including generalising 
to premonoidal categories, are discussed in section [S] 

We will equate sets up to natural isomorphism, which simplifies our notation 
in several places. The unique element of the terminal set 1 will be denoted •, 
in order to avoid overuse of the symbol * which denotes duality. We also have 
deleting and copying functions \ : X ^ 1, A : X ^ X x X. 

A relation R C A x B will be denoted R : A — 1 -!> B, as is usual in the string 
diagrams literature (for example in |19|'). 


2 The category of pregames 

Definition 1. Let X, Y, R and S be sets. A pregame Q : X ® S* ^ Y R* 
consists of the following data: 

• A set Eg of strategy profiles 

• A play function ¥g : Eg ^ Y^ 

• A coplay function Cg : Eg —>■ 

• An individual rationality relatioiQ Eg : Eg — 1-5> X x R^ 

Formally we will have a category whose objects are pairs of sets, and Q will 
be a morphism {X, S) —>■ (F, R). We will suggestively write this as fy : X:^S* —>■ 
Y (g) i?*, as though we had a compact category. Since we do not, this is only 
syntactic shorthand. We extend this notation in obvious ways, for example 
writing A® B* ® C* ® D for {A x D^B x C). We will draw Q using string 
diagram notation as 


S 

R 



X 

Y 


The most interesting part of the definition of a game is the type X x RX . 

^The term ‘individually rational’ is used in economics, in particular in mechanism design, 
where it is an example of a rationality condition that is both ‘individual’ or ‘local’ (applies to 
each player individually rather than a group), and is qualitative rather than quantitative. 
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A pair {x, k) : X x will be called a context, where x is the histor-^ and k is 
the continuation. The continuation represents ‘closing the loop’ between Y and 
R, with arbitrary future computation done by the environment collapsed into 
a single function. The composition and tensor product of pregames operate by 
taking an existing continuation and extending it both forwards and backwards 
in time, as is usual when programming with delimited continuations. 

Also notice the curious similarity between the types of the play/coplay func¬ 
tions and the witnesses used in the Dialectica interpretation of intuitionistic 
implication [3]. Because of this, there is a strong resemblance between parts 
of the proof of theorem [T] and parts of the soundness proof for the Dialectica 
interpretation. 

Definition 2. A closed pregame is a pregame of the form Q : S* ^ Y, with 
X = R = 1. In this case we have X x RX = 1, and so Eg is a unary relation 
on strategy profiles Eg. If a : Tig is a strategy profile for a closed pregame Q we 
will call a an equilibrium of Q iff aEg». 

In a string diagram, the identity object of a category is denoted by empty 
space. Therefore a closed pregame is of the form 


S 



-Y 


We will build pregames, using categorical composition and tensor product, 
from three atomic components: decisions, computations and the teleological 
Broadly, the first two divide a game into players (or agents) and rules. 
The teleological unit is used to model the backward-causality present in game 
theory caused by players reasoning about future events. 

Definition 3. A decision is a pregame Q ■. X ^ Y ® R* satisfying Eg = Y^ 
and Pgcr = a. 

^The generalisation beyond the category of sets is not completely obvious here: for a game 
with side-effects modelled by a monad M on a locally small category the set of contexts should 
be hom(l, MX) X hom(y, MR). That is, the ‘history’ is not a concrete history, but an object 
representing the computation of a history. For example if we have mixed strategies then we 
need a probability distribution over possible histories. 

® Teleology is the form of causality due to agents ‘striving’ to reach some future aim. The 
use of this name here is intended to highlight that this operation alone accounts for a large 
difference between physics and game theory. 
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Graphically, a decision is of the form 


R 



Y 


To specify a decision is to specify an individual rationality relation Eg : 
—H- X X R^. As for selection functions, this relation can encode a large 
amount of information about a player’s preferences and personality. The intu¬ 
ition is that the decision Q represents an agent who observes the value at X 
and makes a choice at T, while reasoning forwards in time about the outcome 
at R. The relation aMg{x, k) should hold iff the agent has no incentive to uni¬ 
laterally deviate from the strategy cr, in the context in which the history is x 
and the outcome resulting from the choice y is ky. For example, for a classical 
utility-maximising agent we have i? = K and 

aE,g{x,k) k{ax)=ma7iky 

y.Y 

More generally, in section |T] we show how to design such relations using multi¬ 
valued selection functions or quantifiers. 

Definition 4. Let f : X ^ Y be a set-theoretic function. We can view f as a 
pregame either covariantly as f : X ^Y, or contravariantly as f* : Y* —>■ X*. 
In both cases we have S = 1, and define •E(a;, k) to be true for every x and k. In 
the covariant case we setFf = f, and in the contravariant case we set C/** = / 
(or, adding an explicit type isomorphism for readability, C/**(*,x) = fx). 

Covariant and contravariant computations are respectively drawn as 



A particularly important example of a computation is the copying compu¬ 
tation Ax ■ X ^ A 0 A. This allows us to use a value more than once, and 
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will be drawn 



Definition 5 . Let Q ■. X ® T* Y ® S* and % : Y ® S* ^ Z ^ R* he 
pregames. The composition TL o Q ■. X ®T* ^ Z ® R* is defined by 

• ^-HoG = YgX Y-u 

• Pwo£;(o'l.O- 2 ) = IPwO -2 oP^CTi 

• CnoGicri,a2)ix,r) = Cgai{x,Cy,o-2i^gcrix,r)) 

• {o’i,a2)^'H.og{x,k) iff aiEg{x,k') and a2^'H0^g^ix,k), where 

k'y = C-H<J2iy, k{V-H(J2y)) 

Definition 6. Let Q : Xi ^ Rl —^ Yi (g) S'* and TL : X2 Z> S2 ^ Y2 'Z R2 be 
pregames. The monoidal product 

g®n:Xi®Sl®X 2 ®S*^Yi®Rl®Y 2® R*2 

is defined by 

• '^g®'H = Yg X Y-u 

• '^g®n{<Xl,<X2){xi,X2) = (PeCTiXi,P'HCT2X2) 

• Ce®w('^i)Cr2)((xi,a;2), (ri,r2)) = ifCgai{xi,ri),C-H(X2{x2,r2)) 

• {ai,a2)^g®-H{{xi,X2),k) iff ( 7 {Eg{xi,ki) and CT2 E'h(x2, ^2) where 

kiyi = (tti o A:)(j/i,P-Hcr2a;2) 

^ 22/2 = ( 7 r 2 o fc)(Pe(71X1,2/2) 

Composition and tensor product are graphically represented, as usual, by 
end-to-end and side-by-side juxtaposition. 

Definition 7 . Teleological unit is the pregame tx '. X ® X* —>■ 1 given by 
Yrx = 1 , Crx • (x, •) = X and with *£1-^ (x, •) holding for all x. 
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We graphically represent the teleological unit by a cup 



This notation is usually used for the unit of a compact structure, which comes 
with a dual cap representing the counit, satisfying the intuitive ‘yanking equa¬ 
tion’. Although we do not have a cap, the notation is justified by theorem 

13 


3 Coherence theorems 


Theorem 1. There is a category Pregame whose objects are pairs of sets 
{X,R), written X 0 R*, and whose morphisms are pregames. The identity 
morphism on X ® R* is idx®/?* = idx <8)idfl, and the composition is the one 
given in the previous section. 

Proof. We begin by noting that the identity pregame is explicitly given by 

• ^idxig)K» 1 






^idxigiK* * idx 


• k for every k 

(Note that idx®/?* is really an endomorphism of {X x 1,1 x i?), but we are 
equating sets up to natural isomorphism.) 


Left identity Let Q ■. X ® S* ^ Y ® R*. We prove that idvig,/?* oQ = Q. We 
have 

• oS = Sidyg„. xYg = lxYg = Yg 

• PidygB* = Pidy»K. • o Pgcr = idv oPqCr = P^CT 

• ‘Cidy»K. ogcr{x, r) = <Cg(T{x, Cidyg,H. • (Pecra;, r)) = <Cga{x, r) 

• aEidy^j^.,og{x,k) aEg{x,k') A aEidY^^.,iVgax,k) <5=^ aEg{x,k') 

where 

k'y = CidygB. • (j/> • y)) = ky 

Right identity We prove that Q o idxigis* = G- We have 

• Sgoidxgs* =^g X =Eg xl = Eg 

• PeoidygB* ^ = 1*0^ ° Pidx®s- • = ° idx = Ega 

• Cgoidy«„.CT(a;,r) = Cid^^g. • (x, C£;CT(Pidxgs. •x,r)) =Cga(x,r) 

• CTE^oidygR. •IEidx®s*(a;,fc')Ao-Eg(Pid^gg.*a;,fc) aEg{x,k) 
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Associativity Let Q : X ® U* —>■ Y ® T*, % : Y ® T* —>■ Z ® S* and 
I: Z(Z)S* (Z)R*.Y\fe have 

^(lon)og = Yg X Yxo-h = Yg x Y-h X Yx = Y-uog xYx = Yxo{nog) 

For the play function we have 

P(IoW) oC (cti , (72, (73 ) = Plow ((72 , (73 ) O Pg; CTi 
= Px(73 O P'H(72 O ¥gai 
= Px(73 oP-Hoc((7i,(72) 

= Plo(Woe)((^li (^ 2 , ( 73 ) 

and for the coplay function have 

C(xow)oe((7i.(^2,(73)(a;,7) = Cgai(x,Cxo'H(cr2,(J3)0Pg<^ix,r)) 

= Cgai(x, Cw(72(Pe(7ix, Cx(73(Pw(72(Pe(7ia;), r))) 

= Cgai(x, Cn<T 2 (Pgaix, Cx(73(Pwog((7i, ( 72 ) 3 ;, r))) 

= Cwoc ((7i, ( 72 ) (x, Cx( 73 (Pwoe (cTi, CT 2 )a;, r)) 

= CxoCHoC)(( 7 i, ( 72 , ( 73 )(a;, r) 

For the equilibrium condition we have 

((7l, (72, (73)E(XoW)oe(3^> ^ 3 ) 

aiRg{x, ki) A ((72, (73)Exow(Pe(7ia:, k^) 

aiEg{x, ki) A (72E^g(Pg(7ix, ^ 2 ) A (73Ex(Pw(72(Pe(7ia;), ks) 

((7i,(72)Eg;ow(a:, fe) A(73Ex(PwoG((7i,(72)a;, ^ 3 ) 

((7l, (72, (73)Exo(Woe)(3;) ^ 3 ) 

where 


k 2 Z = <Cx{z, kz(¥x(T3z)) 

kiy = Cxow(( 72, (73)(y, A:3 (Pxow(c(2, ( 73 ) 2 ;)) 

= Cxow((72,(73)(y, fc3(Pi(73(Pw(72a:))) 

= CHO-2{y, Cx(73(Pw(72y, fc3(Px(73(Pw(72a;)))) 
= Cw( 72(2/, fc2(Pw(72a;)) 


□ 

Theorem 2. The category Pregame is symmetric monoidal, with unit I®!*, 
and the tensor product given in the previous section. 

Proof. We must prove the existence in Pregame of natural isomorphisms Ax : 
7 ® A X, px '. X ® I X, ax,Y,z '■ {X ® Y) ® Z X ® {Y ® Z) and 
ax,Y '■ A®y —>■ Y ®A making certain diagrams commute. Since we are treating 
sets up to natural isomorphism we can take each of these to be the identity, and 
the diagrams all commute automatically. □ 



Theorem 3. If f : X ^ Y is a computation i/ien Tyo(/{g)idy) = rxo(idx ®/*)- 
In diagrams, 




Proof. Firstly note that we have 


•X = fx 
IPidx 0f- •X = X 

We have ^rvo(/ 0 idy) = 1 = ^rxo(idx 

) ^X = Pry • (P/0id^ •x) = • = I 


C/®id^ • ix,y) = y 

^idx 0/* * (^5 ^ 

For the play function, 

Tx • (^idx 0/* *x) = PTxo(idx 0/*) 


• a; 


and for the coplay function, 


Cryo(/®id^) • {x, •) = C/®id^ • {x, Cry • (P/0idJ. • X, •)) 

= Cry • (P/0id^ • X, •) 

= P/®id^ • X 
= fx 

= /(Pidx 0/* • X) 

= /(Crx • (Pidx 0/* • X, x)) 

— Cidx 0/* {x, Crx • (I*idx 0/* * X, *)) 

— *^Txo(idx 0/*) * (^5 *) 

Finally for both cases we have •E(a:, •) for all x. □ 


4 Relationship to selection functions 

In this section we show that pregames subsume certain classes of ‘higher-order 
games’. Firstly we consider the ‘context-dependent games’ of [14], which provide 
a large generalisation of simultaneous games with pure strategies. Secondly we 
consider finite generalised sequential games in the sense of |S], which provide a 
large generalisation of extensive-form games of perfect information. In each case 
we will focus on two-players games for simplicity. We can also easily generalise 
to mixed strategies, which is described in section [Sj 

A two-player context-dependent game is defined in [14] to consist of the 
following data: 
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• Sets X, Y of choices for each player, and E of outcomes 

• Multivalued selection functions e : (X —^ i?) —5- VX, 6 : {Y —5- i?) —> VY 

• An outcome function q : X xY ^ R 

A strategy profile is simply a pair (ai, 0 - 2 ) : X xY. A strategy profile is called 
a selection equilibrium if 

(Ti G eXx.q{x, 0 - 2 ) 

(72 G 6Xy.q{ai,y) 

Theorem 4. The selection equilibria of this game are precisely the equilibria of 
the string diagram 



where 

(TiEpj(*,A:) CTi G efc 

(T 2 EP 2 (•,/;:) (72 G (5fc 

Proof. Algebraically, this string diagram is 

rp o (g (g) Ap) o (Pi (g) P2) 
Unwinding the definitions, we have 


((7i,(72)Etko(9®AJ()o(Pi®P2)(», •) 

(cti, Cr2)Epi0P2(•, k) A •E^H0(g®A^)(lPPi®P2(o-i, < 72 )*, •) 

((7i,(72)Epj®P2(*,A:) 

CTiEp^ (•, fci) A (72Ep2 (•, ^ 2 ) 

(7i G e/ci A {72 G (5^2 


where 


k{x, y) = C^„o(g®A^) • ((x, y), •) 

= Cg^AJ; • {{x, y), Cra • (IPg^AJj • {x, y), •)) 
= Cg®A^ • {{x, y), Cra • {q{x, y), •)) 

= Cq^Aj; • {{x,y),q{x,y)) 

= ^R{q{x,y)) 

= {q{x,y),q{x,y)) 
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kix = (tti o k){x, Pp2CT2») = (tti o fc)(x, 0-2) = q{x, (72) 
k 2 y = (772 o /c)(PpiCri*,y) = (772 o k){(Ti,y) = q{ai,y) 

□ 

A two-player sequential game is defined in [8] to consist of the following data: 

• Sets X, Y of choices for each player, and R of outcomes 

• Multivalued quantifiers (p : {X ^ R) ^ VR, ip ■. (Y ^ R) ^ VR 

• An outcome function q : X x Y ^ R 

A strategy profile for this game consists of a move tri : X for the first player 
and a contingent strategy <72 '■ X ^ Y for the second player. A strategy profile 
is called optimal if 


q{(Ji,a2<7i) G ipXx.q{x,a 2 x) 

q{x,a 2 x) G 5\y.q(x,y) for all x : X 

(Note that the difference between selection functions and quantifiers is relatively 
unimportant: we could equally well define simultaneous games using quantifiers, 
and sequential games using selection functions.) 

Theorem 5. An optimal strategy profile for this game is an equilibrium of the 
string diagram 



where 


(TiEpi(*,fci) kiai £ tpki 

a2Ep^{x,k2) k2{<72x) € 1pk2 
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Proof. Algebraically this string diagram is 

Tfl; o (g (g) AJj) o (((idx ®^ 2 ) o Ax) 0 idfl) o Pi 
Unwinding the definition, we have 


(ui, Cr 2 )]ETj^o(g'®A|j)o(((idx ®P2)oAx)®id|j)oPi (•; •) 


((Ti, Cr2)]E(((idx ^P2)oAx)0id}^)oPi {•, ki) 

0 'lEp^(», ^ 2 ) A Cr2E((idx ®P2)oAx)®id|j(]PPiO'l*j ki) 
o'iEpj(», ^ 2 ) A cr2E(idx 0P2 )oAx fca) 
aiEp^(»,k2) A(T2Eidx ®P2 (Eax •^uks) 
tTiEp^(*,fc2) Acr2Ep2(cri,fc4) 

g(cri,cr20-i) e ipXx.q(x,a2x) Aq{ai,a2ai) G ipXy.q{ai,y) 


where 


ki{x,y) = {q{x,y),q{x,y)) 

k2X = C((idx (giP2)oAx)®idJj'^2(3^) Al(E((idx (giP2)oAx)0id|j'^23^)) 
= ( 7 r 2 o fci)(P('(-id^ (g)P2)oAx)®id|jO'2a;) 

( 7 r 2 O fcx)(E(idx 0 P 2 )oAx^ 2 ^) 

= ( 7 r 2 O fcl)(Pidx 0P2(^2(x,x)) 

= {t^2 ofci)(Eidx •a:^,Pp2 0-2a;) 

= ( 7 r 2 o fci)(x, 0-22:) 

= q{x, a2x) 

k 3 ix,y) = (tti o ki){x,y) 

= q{x,y) 
kiy = fc3(Pidx 
= q{<xi,y) 


□ 


The converse does not hold, because the definition of optimal strategy pro¬ 
file generalises subgame-perfect equilibria, which is an equilibrium refinement of 
Nash [^. The definitions given in this paper generalise easily to subgame-perfect 
equilibria, but we do not do this because subgame-perfection, unlike Nash, is 
not decidable in general. 


5 Future directions 

The potential applications of a compositional, graphical game theory are nu¬ 
merous, especially in economics, and this paper also raises some interesting 
theoretical questions. We conclude by broadly giving some future research di¬ 
rections and questions, most of which are being explored by the author together 
with the other authors of [T^ . 
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• Possibly the most important theoretical concepts missing from this paper 
are repeated games and incomplete information games, both of which are 
ubiquitous in economic applications. This is work in progress. 

• Another important aspect of game theory that cannot be modelled in this 
way is the ability for the ‘shape’ of a subgame to depend on a previous 
move, for example with the ‘moves of nature’ used in the usual approach to 
incomplete information. The obvious approach to this is to use dependent 
types, which leads immediately into current research on type systems. A 
complication is that mechanical type inference has already proven invalu¬ 
able in practice. 

• A promising approach to the semantics of the teleological unit is a strong 
resemblance to shift/reset operators for programming with delimited con¬ 
tinuations mm- a decision is analogous to the shift operator in that it 
captures a continuation, and the teleological unit is analogous to the reset 
operator in that it delimits a continuation. 

• Due to the difficulty of reasoning with continuations, computer support is 
vital for all but the most trivial applications. The author has developed a 
Haskell implementation, but it is extremely awkward to use because the 
Haskell type system does not unify types like A ^ 1 x A and A ^ 1 ^ A, 
and so the user must manually keep track of these isomorphisms. As an 
intermediate step, a code generator for a domain specific language simi¬ 
lar to Haskell’s arrows [18] would be useful. (Unfortunately, for technical 
reasons it does not seem to be possible to use GHC’s built-in arrow pre¬ 
processor.) Ultimately a graphical interface would be invaluable for these 
ideas to become accessible to working economists. 

• As a by-product of obtaining a compositional theory, we have the ability 
to model preferences of agents which are extremely different to utility 
maximisation or preference relations. This extends a line of work begun 
in which uses fixpoint selection functions to model coordination and 
differentiation. Obvious next steps include modelling bounded rationality 
m and social concerns. 

• A potentially very powerful dimension is to vary the underlying category, 
as discussed in the introduction. The use of ordinary (possibilistic) non¬ 
determinism in game theory is explored for example in niEiiia and mi 
chapter 9], and work in progress by the author suggests that the order 
structure on possibilistic strategies is important. We also have experi¬ 
mental evidence that correlated equilibria [5] appear as a special case by 
using a commutative monad transformer stack in which a reader monad 
gives players read-only access to a shared randomising device. 

• Using noncommutative side-effects is potentially even more rewarding, 
but there is theoretical work to be done on graphical languages for Freyd 
categories. A major aim is to use strategies with mutable states to model 
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learning, and individual rationality relations to specify that a strategy can 
be subjectively rational with respect to current epistemic knowledge, for 
example using methods of epistemic game theory |20] . 
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